Frequentist Inference for Semi-mechanistic Epidemic Models with Interventions (preprint)

The effect of public health interventions on an epidemic are often estimated by adding the intervention to epidemic models. During the Covid-19 epidemic, numerous papers used such methods for making scenario predictions. The majority of these papers use Bayesian methods to estimate the parameters of the model. In this paper we show how to use frequentist methods for estimating these effects which avoids having to specify prior distributions. We also use model-free shrinkage methods to improve estimation when there are many different geographic regions. This allows us to borrow strength from different regions while still getting confidence intervals with correct coverage and without having to specify a hierarchical model. Throughout, we focus on a semi-mechanistic model which provides a simple, tractable alternative to compartmental methods.

K-12 School Teaching Posture Correlates with COVID-19 Disease Outcomes in Ohio (preprint)

At the start of the COVID-19 pandemic, most US K-12 schools shutdown and millions of students began remote learning. By September 2020, little guidance had been provided to school districts to inform fall teaching. This indecision led to a variety of teaching postures within a given state. In this report we examine Ohio school districts in-depth, to address whether on-premises teaching impacted COVID-19 disease outcomes in that community. We observed that counties with on-premises teaching had more cumulative deaths at the end of fall semester than counties with predominantly online teaching. To provide a measure of disease progression, we developed an observational disease model and examined multiple possible confounders, such as population size, mobility, and demographics. Examination of micropolitan counties revealed that the progression of COVID-19 disease was faster during the fall semester in counties with predominantly on-premises teaching. The relationship between increased disease prevalence in counties with on-premises teaching was not related to deaths at the start of the fall semester, population size, or the mobility within that county. This research addresses the critical question whether on-premises schooling can impact the spread of epidemic and pandemic viruses and will help inform future public policy decisions on school openings.

An Open Repository of Real-Time COVID-19 Indicators (preprint)

The COVID-19 pandemic presented enormous data challenges in the United States. Policy makers, epidemiological modelers, and health researchers all require up-to-date data on the pandemic and relevant public behavior, ideally at fine spatial and temporal resolution. The COVIDcast API is our attempt to fill this need: operational since April 2020, it provides open access to both traditional public health surveillance signals (cases, deaths, and hospitalizations) and many auxiliary indicators of COVID- 19 activity, such as signals extracted from de-identified medical claims data, massive online surveys, cell phone mobility data, and internet search trends. These are available at a fine geographic resolution (mostly at the county level) and are updated daily. The COVIDcast API also tracks all revisions to historical data, allowing modelers to account for the frequent revisions and backfill that are common for many public health data sources. All of the data is available in a common format through the API and accompanying R and Python software packages. This paper describes the data sources and signals, and provides examples demonstrating that the auxiliary signals in the COVIDcast API present information relevant to tracking COVID activity, augmenting traditional public health reporting and empowering research and decision-making.

Can Auxiliary Indicators Improve COVID-19 Forecasting and Hotspot Prediction? (preprint)

Short-term forecasts of traditional streams from public health reporting (such as cases, hospitalizations, and deaths) are a key input to public health decision-making during a pandemic. Since early 2020, our research group has worked with data partners to collect, curate, and make publicly available numerous real-time COVID-19 indicators, providing multiple views of pandemic activity in the U.S. This paper studies the utility of five such indicators—derived from de-identified medical insurance claims, self-reported symptoms from online surveys, and COVID-related Google search activity—from a forecasting perspective. For each indicator, we ask whether its inclusion in an autoregressive (AR) model leads to improved predictive accuracy relative to the same model excluding it. Such an AR model, without external features, is already competitive with many top COVID-19 forecasting models in use today. Our analysis reveals that (a) inclusion of each of these five indicators improves on the overall predictive accuracy of the AR model; (b) predictive gains are in general most pronounced during times in which COVID cases are trending in “flat” or “down” directions; (c) one indicator, based on Google searches, seems to be particularly helpful during “up” trends.

Causal Inference in the Time of Covid-19 (preprint)

In this paper we develop statistical methods for causal inference in epidemics. Our focus is in estimating the effect of social mobility on deaths in the Covid-19 pandemic. We propose a marginal structural model motivated by a modified version of a basic epidemic model. We estimate the counterfactual time series of deaths under interventions on mobility. We conduct several types of sensitivity analyses. We find that the data support the idea that reduced mobility causes reduced deaths, but the conclusion comes with caveats. There is evidence of sensitivity to model misspecification and unmeasured confounding which implies that the size of the causal effect needs to be interpreted with caution. While there is little doubt the the effect is real, our work highlights the challenges in drawing causal inferences from pandemic data.

Evaluation of individual and ensemble probabilistic forecasts of COVID-19 mortality in the US (preprint)

Short-term probabilistic forecasts of the trajectory of the COVID-19 pandemic in the United States have served as a visible and important communication channel between the scientific modeling community and both the general public and decision-makers. Forecasting models provide specific, quantitative, and evaluable predictions that inform short-term decisions such as healthcare staffing needs, school closures, and allocation of medical supplies. In 2020, the COVID-19 Forecast Hub (https://covid19forecasthub.org/) collected, disseminated, and synthesized hundreds of thousands of specific predictions from more than 50 different academic, industry, and independent research groups. This manuscript systematically evaluates 23 models that regularly submitted forecasts of reported weekly incident COVID-19 mortality counts in the US at the state and national level. One of these models was a multi-model ensemble that combined all available forecasts each week. The performance of individual models showed high variability across time, geospatial units, and forecast horizons. Half of the models evaluated showed better accuracy than a naive baseline model. In combining the forecasts from all teams, the ensemble showed the best overall probabilistic accuracy of any model. Forecast accuracy degraded as models made predictions farther into the future, with probabilistic accuracy at a 20-week horizon more than 5 times worse than when predicting at a 1-week horizon. This project underscores the role that collaboration and active coordination between governmental public health agencies, academic modeling teams, and industry partners can play in developing modern modeling capabilities to support local, state, and federal response to outbreaks. f